Learning Traffic Light Control Policies

Introduction With the growth of modern cities and the reliance of many of their populations on personal automobiles for the primary mode of transport, finding improved means to control the flux of vehicles has grown increasingly important. There are substantial benefits to be derived from improved traffic flow. For many commuters, reclaiming part of their day would enhance their quality of life and less congestion would engender fewer accidents, saving lives. Furthermore, time spent traveling to and from work is not time spent doing work. In fact, most people are essentially constrained to perform only the task of driving as they commute. Goods must be transported and service providers must travel to their clients. Clearly, traffic delays impinge on productivity and economic efficiency. There is also the concern of pollution as cars are generally less efficient in stop-and-go than in smooth flowing traffic. Longer commutes also mean longer running times and entail more greenhouse gases. By improving the policies that control traffic lights, traffic flow can be improved to mitigate these problems and it can be done for considerably less cost than other infrastructural improvements such as increasing the capacity and number of roadways or adding public transit systems. Currently, there are efforts to create reasonable control policies, but most are ad hoc and constitute manual tuning. The result is that drivers notice policies that are clearly suboptimal. We will show that, by applying machine learning techniques, we hope to derive policies that are, at least, locally optimal and are, in the general case, better than manual tuning. These policies would yield a net improvement in the efficiency of traffic systems while maintaining fairness. Since engineers are no longer hand-tuning policies, automated policy search could also yield a reduction in cost of system design.